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Abstract — A conflict detection and resolution tool, Terminal-area 
Tactical Separation- Assured Flight Environment (T-TSAFE), is 
being developed to improve the timeliness and accuracy of alerts 
and reduce the false alert rate observed with the currently 
deployed technology. The legacy system in use today, Conflict 
Alert, relies primarily on a dead reckoning algorithm, whereas T- 
TSAFE uses intent information to augment dead reckoning. In 
previous experiments, T-TSAFE was found to reduce the rate of 
false alerts and increase time between the alert to the controller 
and a loss of separation over the legacy system. In the present 
study, T-TSAFE was tested under two meteorological conditions, 
1) all aircraft operated under instrument flight regimen, and 2) 
some aircraft operated under mixed operating conditions. The 
tool was used to visually alert controllers to predicted Losses of 
separation throughout the terminal airspace, and show 
compression errors, on final approach. The performance of T- 
TSAFE on final approach was compared with Automated 
Terminal Proximity Alert (ATPA), a tool recently deployed by 
the FA A. Results show that controllers did not report differences 
in workload or situational awareness between the T-TSAFE and 
ATPA cones but did prefer T-TSAFE features over ATPA 
functionality. T-TSAFE will provide one tool that shows alerts in 
the data blocks and compression errors via cones on the final 
approach, implementing all tactical conflict detection and 
alerting via one tool in TRACON airspace. 
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i. Introduction 

Managing terminal area traffic is challenging due to the 
density of traffic and complexity of trajectories and separation 
standards. Conflict Alert is a short time-horizon conflict 
detection tool currently in operational use in both the en route 
and terminal area in the U.S. National Airspace, but it is often 
inhibited or desensitized in the terminal area because it 
generates a high number of false alerts. Similarly, The Short 
Term Conflict Alert (STCA) in the Belgian Military airspace, 
Semmerzake (see Appendix D. of [1]) has rendered STCA 
ineffective due to high number of nuisance alerts. These tools 
help maintain safety in the current system, sometimes at the 


expense of capacity, because controller workload is often 
considered as a limitation to capacity [2] . 

Conflict Alert uses only dead reckoning to determine when 
aircraft are in dangerous proximity to each other as compared 
to alerting for losses of separation. Tang et al. [3] augmented 
the legacy dead reckoning approach with flight trajectory intent 
information to create a short time-horizon tool called Terminal- 
area Tactical Separation Assured Flight Environment (T- 
TSAFE). The tool was developed to address the inadequacies 
of Conflict Alert (CA), which is used in the current air traffic 
control milieu. A comparison of T-TSAFE against a model of 
Conflict Alert, in a fast-time environment, found that T-TSAFE 
reduced the false alert rate from 20 per hour to 2 per hour when 
altitude intent was available and provided an average alert lead- 
time of 38 seconds [3]. 

There are several conflict detection tools in the current 
National Airspace such as Conflict Alert (CA), Minimum Safe 
Altitude Warning (MS AW) system and Automated Terminal 
Proximity Alert (ATPA) in the final approach that do not 
communicate with each other. It would be ideal to have a single 
tool throughout the terminal airspace including the final 
approach. This research aims to test the T-TSAFE algorithm 
for depicting alerts in the data block and providing for a visual 
graphic for showing compression errors in the final approach. 

The objective of the current work was to test the T-TSAFE 
algorithm driving the visual graphic similar to ATPA in the 
final approach phase, and also vary the presence or absence of 
aircraft in the traffic flying under visual flight rules. ATPA 
shows the final approach controller a visual graphic (cones) for 
separating two aircraft flying straight in on final, and provides 
warnings when separation is predicted to be lost. 

The paper describes previous research in the field followed 
by a section that compared the T-TSAFE cones with the ATPA 
cones. It continues to delineate the experimental conditions and 
the relevant results are followed by discussion and conclusions. 



II. BACKGROUND 

The Federal Aviation Administration (FAA) has forecasted 
an increase in air traffic demand that may see traffic more than 
double by the year 2025 [4] [5]. Increases in air traffic will 
burden the air traffic management system, and higher levels of 
efficiency will be required. Maintaining current levels of 
safety will be more difficult in a more constrained and crowded 
terminal airspace. Thus, automation is proposed to aid the 
terminal area controllers with the task of assuring separation. 

Terminal airspace has proven to be difficult for tactical 
conflict detection automation. The factors that contribute to this 
difficulty include dense traffic, frequent large turns made by 
aircraft, imprecise flight plans, a complex set of separation 
standards and the fact that the aircraft operate close to the 
minimum separation standards leading to compression errors 
(horizontal separation violation) on approaches [3]. In the 
current-day environment, CA, a legacy system, was shown to 
be inaccurate at times, and have a high rate of false alerts [6]. 
An analysis of Conflict Alert showed that in the terminal area, 
controllers respond to alerts 56% of the time [7], which 
suggests a high false or high nuisance alert rate. 

Eurocontrol has been developing guidelines for a Short 
Term Conflict Alert (STCA) [1]. The guidelines define 
environment data and parameters that should be used for 
conflict detection such as type of flight, wake category, 
Reduced Vertical Separation Minima (RVSM) status, cleared 
or block flight levels, and manually entered flight levels. The 
STCA guidelines propose using the linear prediction filter 
(dead reckoning) with a look-ahead time along with cleared 
flight levels entered by the controller, when available. 

The new algorithm for tactical conflict detection (T- 
TSAFE) developed by Tang et al. [3] aims to address the 
inadequacies of Conflict Alert in terminal airspace and 
incorporates some of the recommendations made by Friedman- 
Berg et al. [9], e.g., using a single analytic trajectory that takes 
into account both flight intent information and the current state 
of the aircraft. It also follows several of the Eurocontrol’s 
STCA guidelines such as taking into consideration different 
route structures, using flight intent information from the 
airspace definitions, speed restrictions and Area Navigation 
(RNAV) departure routes, segments of nominal TRACON 
routes and manually entered altitude clearances by controllers. 

Tang et al. [3] compared the T-TSAFE algorithm with a 
model of the Conflict Alert algorithm using recorded data from 
Dallas/Fort Worth TRACON that included 70 operational 
errors between January 2007 and April 2009. An analysis of 
fast-time simulation data showed that T-TSAFE would have 
prevented most of these operational errors, and that T-TSAFE 
also yielded a false alert rate of 2 per hour with 38 seconds of 
lead alert time, giving the controller more time to address 
conflict situations before they became critical. This finding 
addresses another Eurocontrol STCA guideline, which is to 
provide an alert with ample time to conflict so that the 
controller has enough time to de-conflict the two aircraft. In 
addition, when the algorithm has information about where and 
which aircraft will level off, fast time analyses showed further 
significant reductions in false alerts. The potential benefit from 
additional altitude intent information was the rationale for 


asking controllers to enter some commanded altitudes in the 
current investigation. This is similar to Eurocontrol’s 
guidelines for Cleared Flight Levels (CFL) for the STCA [1], 
where controllers are expected to enter the assigned altitudes 
manually. 

Subsequent to T-TSAFE ’s fast-time study [3], several 
HITL studies also tested T-TSAFE in the terminal airspace 
[8][[9]. Results provide additional evidence for T-TSAFE’s 
efficacy, and the possible usefulness of altitude entries made by 
the controller. The initial HITL study [9] examined controllers’ 
usage of the ATP A cones, which were used to automatically 
depict minimum separation between the aircraft on final 
approach, and the next HITL study [8] compared cones driven 
by the ATP A algorithm to T-TSAFE alerts in the data block 
(Fig. 1). Either the ATPA cones were shown to provide 
warnings or the T-TSAFE alerts were shown in the data block 
of the aircraft on final approach. The Federal Aviation 
Administration (FAA) recently fielded ATPA during the final 
approach phase of flight [10]. The tool provides controllers 
with visual warnings if the minimum separation criteria is 
being exceeded or has the potential for being exceeded. The 
controllers give commands to pilots to make necessary 
maneuvers. ATPA may also allow the controllers to achieve 
better arrival rates by maintaining precise separation between 
aircraft. The tool shows the controller a cone, whose narrow 
end is placed on the aircraft icon and its length is based on the 
required separation between the two aircraft flying in-line (Fig. 
3). 

Currently there are several conflict detection tools in the 
terminal area such as CA, MSAW, ATPA and each of these 
tools acts independently. Allendoerfer & Friedman-Berg 
(2012) discuss that these tools are not integrated, so they do not 
exchange inputs or outputs, and sometimes provide 
contradictory information to the user [11]. They also found that 
tools developed in a non-integrative manner interfere with each 
other and require additional effort for controllers to manage the 
system. T-TSAFE was extended to include cones on final 
approach to integrate some of these tools. T-TSAFE attempts 
to improve upon, and bridge any gaps or missing functions in 
CA and ATPA. 


hi. Experimental Tool-T-TS AFE 

The current investigation examined using the T-TSAFE 
algorithm to drive ATPA-like cones and compared them with 
the original implementation of the ATPA cones. The objective 
of the study was to examine final approach operations using 
either ATPA cones or T-TSAFE cones under different 
operating conditions. 

The main tool used in the experiment was T-TSAFE. The 
alerts were shown in the data block and also as cones. There 
were different levels of alerts and they are all described in this 
section. T-TSAFE uses 1000 feet minimum vertical separation, 
wake turbulence lateral separation standards, and a look-ahead 
time of 120 sec to calculate conflicting trajectories. T-TSAFE 
alerts the controllers to a conflict by placing the number of 
seconds to predicted Loss of Separation (LoS) at the end of the 
first line in the data block, and the call sign of the conflicting 



TABLE I. 


Differences between ATPA & T-TS AFE Cones 


aircraft in the third line of the data block, both in yellow or red 
(Fig. 1). There are two levels of alerts annunciated in the data 
block- yellow alerts when the time to predicted loss of 
separation is greater than 45 sec, and red alerts when the time 
to LoS is below 45 sec. If more than one other aircraft is 
involved in a conflict, the third line shows the call sign of the 
aircraft closest to a LoS. The controller can also roll the cursor 
over any aircraft showing a conflict, causing the data blocks of 
all other conflicting aircraft on the display to turn yellow for 
five seconds. 
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Figure 1. T-TS AFE Data Tag (yellow and red alerts) 


The controllers also entered assigned altitude to the system 
and were provided resolution advisories that were either 
altitude or speed advisories. Both Speed and Altitude resolution 
advisories are shown in magenta color to the controller, in the 
second line of the data block. The assigned altitude entered by 
the controllers via keyboard is shown in green color in Fig. 2. 
The controllers were not required to make any entries for speed 
commands they issued to the controllers. As soon as the T- 
TSAFE algorithm receives altitude intent from the controller’s 
input, it no longer detects the conflict and removes it from the 
data block, thus decreasing false alerts. The green assigned 
altitude stays until the aircraft ascends or descends 300 ft above 
or below the assigned altitude. 
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Figure 2. T-TSAFE Altitude Resolution and Altitude Entries 

The current research compared ATPA cones (Fig. 3) with 
T-TSAFE cones (Fig. 4) in the final approach area. The T- 
TSAFE cones have the same visual graphic as the ATPA 
cones, but they also have added time to LoS in the data block. 
Other differences between the ATPA and T-TSAFE cones are 
described in Table I. 

As mentioned in Table I, ATPA cones appear on the 
aircraft only after the aircraft are established on the localizer, 
and the cones provide alerts in the form of yellow and orange 
cones for compression errors only. The warning yellow and 
orange ATPA cones did not also display T-TSAFE alerts to 
avoid confusion and clutter. This phase required controllers to 
enter the level-off altitudes (issued to the pilots as verbal 
commands) via keyboard command for the purpose of reducing 
the number of false alerts. These altitude entries had no effect 
on the ATPA cones. 


ATPA Cones 

T-TSAFE cones 

Cones show compression error 

Cones show Loss of Separation (LoS) 

Cones appear after established on 
localizer 

Cones appear 30 sec before being 
established on localizer 

Altitude Intent has no impact 

Altitude intent reduces false alerts 

Yellow warning cones when 
predicted time to LoS is 45 sec 

Yellow warning cones when predicted 
time to LoS is greater than 45 sec 

Orange warning cones when 
predicted time to LoS is 24 sec 

Red warning cones when predicted 
time to LoS is less than 45 sec 

T-TSAFE compression alerts are 
suppressed on aircraft with ATPA 
cones 

Only time to LoS is shown in data 
block on aircraft with T-TSAFE cones 

Conflict predictions between aircraft 
are made with the aircraft physically 
ahead 

Conflict predictions between aircraft 
are based on the schedule to the runway 


The T-TSAFE cones evaluate required separation, thus 
sufficient altitude separation prevents the T-TSAFE cones from 
showing yellow or red alert status. ATPA cones, on the other 
hand, turn yellow or orange if there is a compression error 
despite sufficient altitude separation between two aircraft. The 
cones for T-TSAFE appear 30 sec before aircraft are 
established on the localizer. T-TSAFE builds predictions based 
on scheduled arrival at the runway, allowing for the accurate 
display of compression alerts even when two aircraft are not 
yet physically in line. ATPA cones are built for the trailing 
aircraft only when it is physically behind another aircraft. At 
face value, the ATPA and T-TSAFE cones appear the same 
with the exception of additional time-to-LoS shown in the data 
tag for the T-TSAFE cones for compression errors (Fig. 3 & 
Fig. 4). The color of the T-TSAFE warning cones matches the 
color of the T-TSAFE alerts in the data block (Fig. 3). 

Other similarities between the ATPA cones and T-TSAFE 
cones include: The blue cone shows no LoS or compression 
error; the number displayed inside the cone is the distance 
between the leading aircraft and current aircraft; the size of the 
cone for both ATPA and T-TSAFE is based on wake 
requirements given the aircraft types for both the aircraft. The 
experiment examined T-TSAFE cones with ATPA cones under 
IFR and Mixed operating conditions. The Mixed operating 
conditions involved having part of the traffic on visual 
approach. 



Figure 3. Automated Terminal Proximity Alert 
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Figure 4. T-TSAFE cones (Time to LOS shown in data tag) 


The final approach controller made keyboard entries for 
aircraft that were put under visual approach to runway 24R in 
the Mixed operating conditions. They pushed the keys “4R” 
on the keyboard to assign the aircraft to a visual approach on 
24R. The scratch pad entries in the data block can distinguish 
between an aircraft on ILS approach and visual approach. The 
aircraft on ILS approach to 24R shows “I4R” and to 25L 
shows “I5L”. The aircraft on visual approach to 24R shows 
“24R” (Fig. 5) and to 25L shows “25L” in the scratch pad of 
the data block. 
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Figure 5. Data tag showing visual entry to 24R 

Another new tool explored in the study was controller look- 
ahead time. The controllers were provided a tool that allowed 
them to set the look-ahead time on the display to control the 
alerts they wanted to see on their scopes. They were allowed 
to select from a drop-down menu - 45, 60, 90 or 120 sec as 
their look-ahead time. For example, a controller could select 
60 sec as the look-ahead time, and all alerts within the 60 sec 
of predicted time to conflict were shown on the display. This 
allowed controllers to control how early and how many 
conflicts they chose to see on their scope. 

iv. Methodology 

The experimental approach was a Human-In-The-Loop 
(HITL) evaluation of the T-TSAFE tools under current-day 
operational conditions. 

A. Experiment Matrix. 

This study phase tested four conditions: T-TSAFE cones 
under Instrument Flight Rules (IFR) and Mixed operating 
conditions, and ATPA Cones under IFR and Mixed operating 
conditions. Four traffic scenarios were exercised with each of 
the conditions for a total of 16 runs. The two variables being 
manipulated are cone type-T-TSAFE and ATPA, and 
operations type- IFR and Mixed. 

B. Traffic Scenarios 

The study simulated five arrival streams to and one 
departure stream from Los Angeles International Airport 
(LAX) using current airspace and procedures within the 
TRACON. The scenarios were designed to create situations 
that would result in a LoS between aircraft unless a controller 
intervened. Occasionally, the controllers were able to 
successfully avoid conflicts for extended periods due to early 
interventions. It was therefore necessary to add conflicts to the 


scenarios using observers collaborating with pseudo pilots. 
Each scenario involved heavy current-day traffic, with all LAX 
traffic under Instrument Landing System (ILS) simultaneous 
approaches on runways 24R and 25L for Conditions A and B. 
In the Mixed conditions (Condition C and D), some aircraft 
approaching runway 24R managed by the Stadium controller 
operated under visual flight rules, while the rest of the traffic to 
runway 24R and all approaches to runway 25L operated under 
Instrument Flight Rules (IFR). 


C. Experiment Procedures 

The study was conducted over a two-week period with two 
teams of controllers, each participating for a week. Eight 
recently retired controllers served as participants, each 
controlling simulated traffic in the Southern California 
Terminal Radar Approach Control (TRACON) for 
approximately eight hours total. Each controller team consisted 
of four controllers that had retired less than two years ago from 
Southern California TRACON. Both controller teams were 
briefed on the T-TSAFE concept, the T-TSAFE interface, and 
the conditions of the study. During each week, the controller 
team completed 16 runs, four runs in each of the four different 
conditions, rotating through four different traffic scenarios. 
Controllers also rotated between sector positions after each run. 
Pseudo-pilots flew all the aircraft in the scenarios. The 
controllers worked the East Feeder and Zuma, feeder sectors, 
and Downe and Stadium, approach sectors in the Southern 
California TRACON, rotating positions after each run. All 
controllers completed questionnaires after every run and took 
part in a debrief session at the end of the study. Both conflict 
detection performance and controller subjective feedback were 
collected and analyzed, to characterize the performance of the 
T-TSAFE prototype. 

v. Results & Discussion 

Both the digital and questionnaire data were collected and 
analyzed for the two independent variables: Operating 
conditions (IFR and Mixed), and Cone types (ATPA and T- 
TSAFE). Only the Stadium controller experienced the Mixed 
condition, therefore, the analysis of the Mixed condition 
pertains to the Stadium controller only. This is compared with 
the IFR condition analysis pertaining to all four controller 
positions. Also, the cones were displayed to only the final 
approach positions-Stadium and Downe, thus their data are 
considered in the data analysis comparing ATPA and T- 
TSAFE under IFR operating condition only. 

A. Total Number of Alerts Generated & Displayed 

The total number of alerts generated by the T-TSAFE 
algorithm versus the alerts that were displayed is shown in 
Fig. 6 for IFR and the Mixed operating conditions. The total 
number of alerts generated by T-TSAFE is based on a look- 
ahead time set inside the T-TSAFE algorithm whereas the 
alerts displayed in the data tag to the controller are managed 
by several factors, some of which are controlled of the 
controller, such as aircraft under visual approach, controller 
look-ahead time, and presence of cones. The alerts displayed 
to the controller impacts the controller’s workload therefore it 






is compared with the Total number of alerts generated. As 
depicted in Fig. 6, T-TSAFE generated significantly larger 
number of alerts under the Mixed condition (average of 17) 
than IFR condition (average of 10) (F (1,126)=15.97, 
p<0.001). The higher number of alerts generated for the 
Mixed operating condition is due to the fact that aircraft under 
visual separation come very close to other aircraft because 
they have no standard separation requirement. It is interesting 
to note that the total number of T-TSAFE alerts displayed is 
considerably smaller than the number of alerts generated. On 
average 8 alerts were displayed during the course of a 40 
minute run on each controller station. The reason for the low 
number of alerts being displayed is because alerts are 
suppressed when aircraft are on visual approach, have ATPA 
cone or are outside a controller’s look-ahead time (discussed 
in the next subsection). All these factors contribute to the low 
number of alerts displayed on the aircraft. 
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Figure 6. Total Number of alerts generated vs. displayed 
(IFR vs. Mixed) 


An ANOVA revealed a significant difference in total 
number of alerts displayed for the visual aircraft ( one on an 
average) versus none for IFR (F (1,1 26)= 17. 3 8, p<0.001). In 
general, most alerts on aircraft under visual separation were 
suppressed, except any alerts with general aviation aircraft 
flying under visual flight rules (VFR aircraft). 

Fig. 7 shows the Total Number of alerts generated versus 
displayed for T-TSAFE cones vs. ATPA cones condition for 
the two final controllers -Stadium & Downe. Both the ATPA 
and the T-TSAFE cone conditions are provided with different 
cones in the final approach, but T-TSAFE alerts show in the 
data tags for rest of the traffic in the airspace. It was found hat 
the cone condition did not impact the number of alerts 
generated by the T-TSAFE algorithm and no significant 
differences were yielded on ANOVA. As mentioned earlier, 
the number of T-TSAFE alerts generated for ATPA and T- 
TSAFE cone condition has the same T-TSAFE algorithm 
working in background and thus the number is quite similar 
(about 15 alerts per run). Fig. 7 also shows that the number of 
alerts displayed in the two conditions is somewhat similar. 
About 12 T-TSAFE alerts were shown in the ATPA condition 
and 15 alerts were shown in the data block of aircraft in T- 
TSAFE cones condition reaching marginal significance on 
ANOVA (F(l,44)=3.4, p<0.07). The reason for this difference 
lies in the fact that T-TSAFE alerts for compression errors are 
suppressed when ATPA cones are depicted on the aircraft, 


whereas the alerts are partially shown on the aircraft with T- 
TSAFE cones, providing the controller with time to predicted 
LoS in the data tag. There was a significant difference in the 
alerts shown on visual aircraft in the ATPA condition as 
compared to T-TSAFE condition (F(l,44)=4.86, p<0.05). It is 
possible that the confederates managed to create more alerts 
with aircraft under visual condition (about 1 more alert) in 
ATPA than in T-TSAFE condition. The only alerts that were 
depicted on the aircraft under visual separation were alerts 
with VFR aircraft. 



Figure 7. Total Number of alerts generated vs. displayed 
(ATPA vs. T-TSAFE cone conditions) 


B. Controller Look-ahead Time 

The controller manipulated look-ahead time was a new 
concept in the study. The look-ahead time inside the T-TSAFE 
algorithm was set at 120 sec. The controller was allowed to 
manipulate the “controller look-ahead,” as it impacted the 
display which allowed the controller to decide how far in 
advance they wanted to display the alerts. Controller 
participants could, for example, choose 45 sec as the look- 
ahead, for which they would see alerts with time to predicted 
LoS below 45 sec. However, the participants often confused 
this to mean that they would get every alert at 45 sec to 
conflict. This means they will have to be trained well to 
accurately use the controller look-ahead time. 

The different analyses show that controllers prefer 60 sec 
as the look-ahead time for getting alerts in their respective 
scopes. Any alerts generated by T-TSAFE with time to 
predicted LoS greater than the controller look-ahead time were 
suppressed by the system and not displayed to the controller. 
There were no significant differences found on a chi square 
test, where the expected frequency of a certain look-ahead 
time was compared with the actual frequency of that look- 
ahead time between IFR and the Mixed conditions. There was 
a similar tendency for the controllers to select 60 sec most 
frequently irrespective of the type of cone - ATPA or T- 
TSAFE (Fig. 8). The look-ahead time can interact with the 
Time-to-Predicted-LoS provided by the algorithm. Similar to 
the T-TSAFE alerts, T-TSAFE cones were also impacted by 
controller’s look-ahead time as controllers did not see any 
warnings for compression errors on T-TSAFE cones unless the 
time-to predicted-LoS was below the controller’s look-ahead 
time. In the case of ATPA cones, since the warning alerts on 
the cones are based on hard-coded numbers i.e. time-to- 


predicted-LoS being 45 sec invoked a yellow cone, and it 
being 24 sec or less invoked an orange cone, controllers’ look- 
ahead time had no impact on the ATP A warning cones. Since 
controller look-ahead time does not impact the display of 
alerts on ATPA cones, this presents integration and training 
issues when ATPA cones and T-TSAFE alerts are integrated. 
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Figure 8. Frequency of different Controller Look-ahead times ATPA cones vs. 

T-TSAFE cones (Downe & Stadium Controller) 

C. Duration of Displayed Alerts 

The duration of the alerts displayed for the IFR conditions 
is compared with the Mixed Condition in Fig. 9. The duration 
of the alerts is longer in the IFR conditions (61 sec compared 
to 44 sec in the Mixed) because in the Mixed condition, only 
the alerts with VFR aircraft are shown. There is no standard 
separation required with VFR aircraft, but the algorithm uses a 
1.5 nmi lateral and 500 ft vertical separation to generate alerts 
between IFR and VFR aircraft. The alert durations did yield 
significant differences between the IFR and the Mixed 
conditions (F(l,1201)=4.98, p<0.05). 

The Duration for the alerts displayed under the ATPA and 
T-TSAFE condition is shown in Fig. 10. The T-TSAFE alerts 
with ATPA are suppressed as soon as the cones appear, which 
explains the lower duration of the T-TSAFE alerts on the 
ATPA cones condition. There was marginal significance 
between the duration of T-TSAFE alerts shown in the ATPA 
cones and T-TSAFE cones conditions (F(l,650)=3.27, 
p<0.07). The duration of the T-TSAFE alerts on the ATPA 
cones is still 60 sec giving the controller enough time to deal 
with them. 
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D. Keyboard Entries 

Controllers made altitude inputs, which are similar to 
cleared flight levels or assigned altitudes, into the system. The 
number of these altitude entries is compared across the 
conditions. It was found that controllers make about two 
altitude entries per run per controller station and there was no 
difference between the IFR and Mixed conditions (Fig. 11). 
The number of altitude entries may be small because the T- 
TSAFE algorithm derives flight intent information from the 
published nominal approaches that aircraft are expected to fly. 
The altitude entries made by the controller in one station are 
shared with the other controller station, providing shared 
situational awareness. Similarly, no difference between the 
altitude entries was observed between the ATPA and T- 
TSAFE cones condition (Fig. 12). 
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Figure 11. Altitude and Visual entries for IFR (all positions) vs. 

Mixed (Stadium) 

The visual operations under different conditions are 
also analyzed for visual entries. These are the “4R” entries 
made via keyboard on aircraft on visual approach (Fig. 5). 
There is a significant difference in the entries made for visual 
approach under the Mixed condition as compared to IFR 
condition (Fig. 11) as indicated by the ANOVA (F (1,126) = 
630.2, p<0.001). This result is expected since visual approach 
entries were allowed only under the Mixed condition. The 
difference in the cones did not impact the visual approach 
entries (Fig. 12). 


Figure 9. Duration of Alerts IFR (All positions) vs. Mixed (Stadium only) 
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Figure 13. Workload IFR (all positions) vs. Mixed (Stadium only) 


Figure 12. Altitude and Visual entries for ATPA cones vs. T-TSAFE cones 
(Downe & Stadium) 

E. Workload & Situational Awareness 

Participants completed the NASA TLX workload 
questionnaire [12] after every run. Data were collected on each 
of the six TLX workload measures. In addition, a seventh 
variable measuring overall workload combining all six of 
these measures was derived. The overall workload variable, 
also known as the “composite” measure, once derived, was 
then scaled down to match the l-to-5 range for direct 
comparison with the other six measures (l=very low, 5= very 
high). Also, the “performance” measure was analyzed on an 
inverse scale, so a higher score indicates lower performance. 
Participants also completed an abbreviated version of the 
Situational Awareness Rating Technique (SART) scale at the 
end of each simulation run [13]. The participants responded to 
questions on the demands the situation posed on them and 
their understanding of the situation offered to the user by the 
displays and procedures. They also responded to a question on 
attention capacity that refers to the user’s skills and attention 
needs. The situational awareness scales ranged from 1 to 7, 
i.e., from very low to very high situational awareness. 

The results on workload (Fig. 13) and Situational 
Awareness (SA) (Fig. 14) show that there were no significant 
differences yielded on the ANOVA when IFR conditions were 
compared with the Mixed. There were subtle trends observed, 
controllers reported somewhat decreased workload under 
mixed operating conditions. Visual operations usually 
decrease workload on the controller because the responsibility 
for separation is delegated to the flight deck. It is interesting to 
note in Fig. 6 that visual aircraft did not get many alerts 
displayed, as they do with the legacy system CA, which can 
potentially increase workload. In fact they received less than 1 
alert per run, which was caused by general aviation aircraft 
under visual flight rules. All other alerts on visual aircraft 
were suppressed. Similar trends are seen with Situational 
Awareness (SA) data; trends show that the demand on 
attention is lower for the Mixed condition. There were no 
differences on the rest of SA variables. 



Demand on attention Level of Attentional Capacity 
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Figure 14. Situational Awareness - IFR (all poistions) vs. 
Mixed (Stadium only) 


The workload reported by the controller participants was 
compared for the T-TSAFE cones and ATPA cones under the 
IFR operating conditions. These were considered because the 
only controller positions that used the cones were the final 
approach controllers. The Mixed operating conditions were 
not included in the analysis because between the two Final 
Approach controllers, only the Stadium controller was allowed 
to have visual separation between some aircraft. Analysis of 
Variance (ANOVA) did not yield any statistical difference 
between the ATPA and T-TSAFE under IFR operating 
conditions on the workload and SA measures. It is worth 
noting that despite the differences between the ATPA and T- 
TSAFE cones, the controllers did not perceive them 
differently in terms of workload or SA. ATPA is currently 
fielded in some of the US TRACONs. The similarities in the 
workload and SA reported for the ATPA cones and TTSAFE 
cones conditions will make bringing T-TSAFE cones in the air 
traffic controllers’ work environment easier in future. The 
controllers were not confused about whether they were 
working with T-TSAFE cones or ATPA because they reported 
that the T-TSAFE cones appeared on aircraft much sooner 
than the ATPA cones. Thus there is anecdotal evidence that T- 
TSAFE cones did provide the controllers better situational 


awareness than the ATP A, but no subjective measures to 
support it. 


cones. This result on additional alert information achieved 
marginal significance with p<0.07 on a binomial test. 


F. Safety & Controlability 

The controller participants were asked to rate operations 
for safety, controllability and acceptability on a scale of 1 to 5, 
where 1 referred to the lower end of the scale and 5, referred 
to the higher end of the scale. The controller participants on an 
average reported high levels on safety, controllability and 
acceptability of the operations. On the questions asked about 
the safety, controllability and acceptability of IFR operations 
as compared to the Mixed operations, no difference in the 
ratings were reported by the controller participants. Similarly, 
no significant differences were reported between the 
operations for ATP A cones vs. T-TSAFE cones for the 
Stadium & Downe controllers. 


H. Complacency in Automation 

A Complacency Potential Rating Scale was used to collect 
data on automation-induced complacency [14]. Wiener [15] 
defined complacency as “a psychological state characterized by 
a low index of suspicion.” Automation is often identified as a 
significant factor that induces complacency. Procedures, roles 
and responsibilities are also potential factors that induce 
complacency. According to Wickens [16], reliability in 
automation engenders excessive trust and over-reliance in 
pilots. Singh et. al. [14] identified four factors that may be 
related to over-trust or complacency in automation. These are 
confidence, reliance, trust, and safety in automation. Some 
examples of scale items that measure different complacency 
constructs are shown in Table III. 


G. Comparison of Features between ATP A cones & 
T-TSAFE cones 

Controllers were also asked rate their preference over the 
features of ATPA and T-TSAFE cones (Fig. 15). These 
features have been described in Table I. Forty percent of the 
controllers preferred the ATPA cones warning for showing 
compression error, whereas sixty percent of the controllers 
prefer that the T-TSAFE cones show LoS, which means that 
altitude separation between aircraft do not result in warning 
cones under T-TSAFE cones, even if lateral separation is less 
than the minimal requirement. The controller’s preference for 
T-TSAFE and ATPA on LoS vs. compression error was 
similar, resulting in a non-significant binomial test. 


HATPA Cones 
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of separation resolutions aircraft 


Figure 15. Controller preference for features of T-TSAFE cones and ATPA 

Among all the other features, The T-TSAFE cone features 
such as their warnings, their early appearance, the impact 
altitude entries have on the cones, and alerts shown for 
merging aircraft were preferred by almost 100% of the 
participants over the ATPA features. All other features yielded 
significantly favorable results for T-TSAFE cones with 
significance achieved at 99% probability level. Only 10% of 
the controllers did not want the additional alerts provided by 
the T-TSAFE, whereas 90% of the participants preferred the 
additional alert information such as the time-to-predicted LoS 
shown on the T-TSAFE cones that was missing in the ATPA 


TABLE II. Examples of statements used to measure 

COMPLACENCY IN AUTOMATION 


Confidence 

T-TSAFE makes air traffic in the terminal environment safer 
under IFR conditions. 

Reliance 

T-TSAFE cones have made the controller’s job easier. 

Trust 

T-TSAFE is more likely to be correct than manual conflict 
detection under visual conditions. 

Safety 

I feel safer having ATPA driven cones than relying on manual 
conflict detection. 


The Complacency Potential Rating Scale was adapted and 
used to collect data for all controller positions across all 
conditions. The scale uses a 5 -point Likert scale that ranges 
from ‘strongly disagree’ to ‘strongly agree.’ Some of the 
questions in the rating scale were reversed to ensure reliability 
in the responses. The scale was adapted to ask questions about 
the IFR, Mixed, ATPA cones, and T-TSAFE cones. 

Fig. 16 shows the controller complacency ratings, 
comparing IFR operating conditions to The Mixed operating 
conditions and comparing the T-TSAFE cones with ATPA 
cones. Statistically significant ANOVA differences were found 
for several constructs of complacency and they have been 
outlined in Table IV. 

The results show that there are significant differences in the 
level of confidence, reliance, trust and safety for operations 
type. IFR operations have consistently higher levels of 
confidence, reliance, safety and trust than the Mixed 
operations. Similarly, there are significant differences in the 
level of confidence and trust for the T-TSAFE cones showing 
that participants were beginning to rely on the T-TSAFE cones 
much more than the ATPA cones. Previous research has 
examined automation complacency [17] [18] and the extent to 
which over-reliance on automation can lead to operational 
errors (e.g., in the case of occasional automation failures). This 
might suggest that very high or very low levels of automation 
complacency are not desirable, but rather, optimum levels of 
automation complacency would be somewhere between these 
two extremes. Thus neither mistrust nor over-reliance in the 
automation is desirable. The high scores on T-TSAFE indicate 
over reliance on the tool and warrants extra caution during 
deployment. 


TABLE III. 


Significant results on various constructs of 
Potential for Complacency 


Construct 

F statistic 
(df values) 

Significance 

Confidence 

F(l,7) = 15.91 

Significant for operation type at p<0.005 

Confidence 

F(l,7) = 4.82 

Marginally Significant for cone type at 

p<0.06 

Reliance 

F(l,7) = 5.90 

Significant for operation type at p<0.05 

Trust 

F(l,7) = 14.97 

Significant for operation type at p<0.05 

Trust 

F(l,7) = 3.94 

Marginally Significant for cone type at 

p<0.08 

Safety 

F(l,7) = 6.67 

Significant for operation type at p<0.05 



Figure 16. Potential for complacency compared across conditions. 


I. Usability Functions 

The controller participants also rated on a scale of 1 to 5 
their subjective responses towards several questions related to 
procedures, information requirements, and usability. 
Significant results were found on the ease of locating aircraft 
with potential LoS, awareness to potential LoS and timeliness 
of alerts. 

The controllers reported that it was easier to locate an 
aircraft with potential LoS with T-TSAFE cones (Fig. 17) as 
compared to ATPA cones (F(l,7)=5.727, p<0.05). This result 
makes sense because the T-TSAFE cones appear sooner than 
the ATPA cones. The controllers also had an easier time 
maintaining awareness of potential loss of separation than the 
ATPA cones (F (1,7)=7.0, p<0.05). This may be due to the fact 
that the T-TSAFE cones have warnings that have additional 
information such as time to predicted LoS shown in the data, 
block. The consistency between the alerts and the warning 
cones also makes for better alerts with T-TSAFE cones than 
ATPA cones. It was also found that T-TSAFE cones performed 
significantly better than the ATPA cones on timeliness of alerts 
(F (1,7)=7, p<0.05). Again the features of T-TSAFE such as 
cones appearing on the aircraft 30 sec before the aircraft is 
established on localizer, and providing warning for aircraft that 
are ahead of the schedule of the current aircraft whereas ATPA 


provides warnings with aircraft that are physically ahead of it, 
and not necessarily ahead in the schedule. Overall, T-TSAFE 
cones performed similar to or better than the ATPA cones. 

There were no significant differences in the ease of locating 
potential LoS between IFR and Mixed operations. Similarly, 
there are no significant differences between IFR and the Mixed 
operating conditions for the responses towards maintaining 
awareness of LoS or on the timeliness of alerts. Thus, the 
operating condition did not impact the nature of alerts or any 
functions related to conflict detection. 



IFR Mixed T-TSAFE ATPA cones 

Operations Operations cones 


Figure 17. Subjective responses on ease of locating aircraft with potential 
loss of separation 

VI. Conclusions 

The study investigated a tactical conflict detection tool (T- 
TSAFE) in the terminal area. This tool combines dead 
reckoning with flight intent where possible and has fulfilled 
many of the requirements set by Eurocontrol’s guidelines for a 
STCA. Initial and follow-up human-in-the-loop air traffic 
control simulation experiments were conducted, to test the 
human factors of this new conflict detection and resolution 
tool, T-TSAFE. Whereas the initial investigation verified T- 
TSAFE as an improvement over the legacy conflict detection 
system currently used in the field, it also revealed aspects of the 
T-TSAFE system that require further investigation [5]. To 
address this need, a follow-up experiment was conducted, 
testing T-TSAFE under four experiment conditions. The first 
two conditions varied the operating conditions- IFR vs. the 
Mixed conditions where aircraft were allowed to have visual 
separation in the latter. The second and third conditions varied 
the software driving the cones - Automatic Terminal Proximity 
Alert (ATPA) and T-TSAFE. The cones are a graphical tool 
used for monitoring final approach and providing a warning for 
possible compression errors in the final approach phase of 
flight. The test bed for the investigation was Simulated- 
Southern California TRACON. 

Workload and Situational differences were not found for 
either cone type or operating type. But trends showed that the 
Mixed operating condition was easier on the controller with 
respect to attentional capacity and workload, because for visual 
operations the controllers delegate responsibility for separation 
to the flight deck. The controller participants preferred to see 
alerts within 60 sec of conflict, even though they had the choice 
of 45 sec, 90 sec and 120 sec as controller look-ahead time. 
They were provided with two levels of alerts - red and yellow 


and used the color of the alert to prioritize their cognitive 
resources. 

Controller participants reported preferring most of the T- 
TSAFE cones’ features to the ATP A cones’ features. They 
particularly liked the fact that T-TSAFE cones appeared 30 
sec before getting established on the localizer making it easy 
to locate an aircraft having potential LoS and to maintain 
awareness of the LoS. They found the alerts provided by T- 
TSAFE as very timely. There was also higher consistency 
between the alerts and cones in case of T-TSAFE cones, when 
compared to ATP A cones. 

For the subjective responses on complacency towards 
automation, T-TSAFE cones were being relied upon more 
than ATP A cones. The Mixed operating procedures are 
keeping the controllers in the optimum zone for complacency 
for automation. 

The comparison between IFR and Mixed operations for T- 
TSAFE has shown that Mixed operations do not increase the 
workload or decrease situational awareness, suggesting that the 
tool is not adding any unnecessary alerts in the Mixed 
condition. Thus, this investigation suggests that having one 
system (T-TSAFE) that shows the alerts in the data blocks and 
also drives the cones in the final approach will circumvent the 
need for integrating different systems. Previous results have 
shown T-TSAFE an improvement over the legacy system. The 
results of this study show similarities between the ATPA cones 
and T-TSAFE cones will make it easier to migrate the air 
traffic control system from ATPA cones to T-TSAFE cones. 
The study also shows that T-TSAFE alerts and ATPA cones do 
to increase undue workload or confusion, thus the transition of 
T-TSAFE alerts with ATPA in the final approach holds much 
potential. 
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